In digital image processing and, in particular, image processing for the purpose of enhancing and optimizing the process of images capture by a camera, it is often useful to automatically detect human faces. Applications of face detection include face recognition, image beautification, image scene balancing, image retrieval, security surveillance and person identification.
A preponderance of images collected by photographers contain human facial images, which are often the most important subjects of the images. Knowledge of the presence and location of faces in an image, and especially the presence and location of the faces, could enable many beneficial improvements to be made in the image capture process. Some are suggested in the prior art. For example, automatic and semi-automatic focusing cameras often pick a portion of the scene on which to adjust for best focus. If the camera could locate the faces in a scene, then focus could be optimized for the faces unless the photographer explicitly overrides that choice. In U.S. Pat. No. 5,835,616, a face detection system is used in automated photography to eliminate manual adjustment problems that can result in poor quality from lack of focused subjects.
Furthermore, detection of the faces in a scene gives very strong evidence of the proper location of the principal subject matter. In that connection, the process disclosed in the '616 patent automatically finds a human face in a digitized image taken by a digital camera, confirms the existence of the face by examining facial features and then has the camera automatically center itself on the detected face. Detection of a face also yields strong evidence of proper color balance for the facial and/or skin area. For example, in U.S. Pat. No. 5,430,809 a video camera autonomously tracks a facial target in order to set a measuring frame on the facial object for purpose of auto exposure and auto focus. In addition, once the measuring frame is set, an auto white balance system adjusts colors to obtain optimal skin color on the face. As a result, the auto white balance system is said to perform auto skin color balance. It is also known (from U.S. Pat. No. 5,629,752) to detect a human face and then to utilize data representing color and/or density of the facial region to determine an exposure amount such that the region corresponding to the face can be printed appropriately by a photographic printer.
A great amount of research effort has been spent on proposing a wide variety of face detection techniques. Face detection is defined as locating the existence of a face in an image. Face recognition on the other hand is defined as assigning an identity to a detected face. Face detection is often a precursor to face recognition. In general, prior art face detection techniques can be classified as either feature-based or region-based. The techniques in the feature-based category first derive low level features such as edges and then use face knowledge-based analysis to find face candidates in an image. The apparent properties of the face such as skin color and face geometry are exploited at different system levels in this category. Feature-based approaches have dominated the face detection research interest for a quite long period before the recent shift of interest to using pattern recognition theory in face detection. It is worth noting that the main components used in the feature-based techniques are edges that come from a high frequency portion of the image and, in general, are not as stable as components from the low frequency portion. Also, in using skin color for face detection, most techniques employ carefully designed skin color models in order to achieve high skin detection accuracy with very low false positives. However, skin color models having a high degree of accuracy often tend to exclude skin colors falling out side of the majority skin color region upon which skin color models are built. The exclusion of non-majority skin colors, in turn, results in face detection failures.
In recent years, benefiting from the advances in pattern recognition theory, techniques in the region-based category treat face detection as a pattern recognition problem without the application of face knowledge. The performance of the methods in this category solely relies on the quality of a classifier trained by learning examples from face class and non-face class training images. The number of samples of face and non-face (or the variation of training images) used in the training process largely determines the effectiveness of the classifier. More training images result in better classifiers. It is a commonly accepted understanding that, while the number of face training images may have a limit, the number of non-face training images has no limits. Thus, repeated training is not uncommon for techniques in this category. Huge footprints (memory spaces required to store trained classifier information in a computer to do the face detection task) may result, which may become a problem for applications with limited computing resources. It should also be noted that face structural (geometry) information is basically not used in techniques in this category.
While face detection has been studied over the past several years in relation to the subject of image understanding, it remains an area with impressive computational requirements, particularly if a robust face detection algorithm is needed. A number of methods have been devised that show reasonable performance over a range of imaging conditions. For example, in a recent article, Human Face Detection in A Complex Background, published in Pattern Recognition, Vol. 1, pp. 53-63, 1994, Guangzheng Yang and Thomas S. Huang proposed a hierarchical knowledge-based method to locate human faces in an image. Their method showed some promising results. However, the method presented a slow execution speed. The challenge is to devise face detection systems that produce acceptable detection accuracy with low computational cost (rapid execution). If this can be done successfully, the detection of faces in a scene will then serve as a springboard to numerous other improvements in the image capture process. In addition, it would be useful to detect faces in order to implement downstream activities after image capture, e.g., face detection could provide evidence of up/down orientation for subsequent printing (for example, of index prints).
It is now well understood that building an automatic face detection system is an very challenging problem due to the fact that there are practically unlimited variations in appearance of human faces in terms of color, posture, size, brightness, and texture, among others. Each of the existing face detection system designs could only accommodate a very small as well as different portion of the variations of face appearance in the ever increasing number of photographic images for various applications. Moreover, for a particular face detection algorithm, high execution speed and high detection accuracy usually do not have a common ground.
There is a need therefore for an improved face detection method that overcomes the problems set forth above.